Claims 

We claim: 

1 1 . A system for receiving audio input comprising: 

2 a display for displaying electronic information; 

3 an audio input receiving audio content; and, 

4 a processor for associating said received audio content with said displayed electronic 

5 information. 

1 2. The system according to claim 1, wherein said audio content is in the form of 

2 audio clips. 

^ 1 3. The system according to claim 1, said processor further associating at least one 

*22 property with said audio content and wherein said audio content is randomly accessible based on 

12 3 said at least one property. 

3 1 4. The system according to claim 3, further comprising: 

M2 a storage for storing said audio content with said at least one property. 

W i 5. The system according to claim 1 , further comprising: 

^ 2 an input receiving a user's input, 

3 wherein said processor starts recording audio content from said audio input in response to 

4 said user's input. 

1 6. The system according to claim 1, wherein said processor includes a voice 

2 activated recording system for recording said audio content. 

1 7. The system according to claim 6, wherein said voice activated recording system 

2 records when said audio content exceeds a predetermined threshold. 
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1 8. The system according to claim 6, wherein said voice activated recording system 

2 records when a known user's voice is detected in said audio content. 

1 9. The system according to claim 1, wherein said processor controls said display to 

2 indicate that audio content is associated with said displayed electronic information. 

1 1 0. A system for playing audio content, said system comprising: 

2 a display for displaying electronic information; 

3 a storage for storing audio content, said audio content including properties and having 

4 been associated with said displayed electronic information; 

5 an output for outputting at least some of said audio content with navigation of said 

6 displayed electronic information; and 

7 a processor for controlling said display, said storage and said output, 

1 11. The system according to claim 10, wherein said audio content are audio clips. 

1 12. The system according to claim 10, wherein said audio content is randomly 

2 addressable based on said properties. 

1 13. The system according to claim 12, wherein said storage is a database. 

1 14. The system according to claim 10, further comprising: 

2 an input for receiving a user' s input, 

3 wherein said output outputs at least some of said audio content in response to receiving 

4 said user's input. 

1 15. The system according to claim 10, further comprising: 

2 an input for receiving a user's input, 

3 wherein said processor searches properties of said stored audio content in response to 

4 said user's input. 
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1 16. The system according to claim 15, wherein the output of said processor is sent to 

2 said display to display an indication of the search results. 

1 17. The system according to claim 15, wherein the output of said controller is sent to 

2 the output for playing audio content with properties matching the search results. 

1 18. The system according to claim 10, wherein said processor retrieves all audio 

2 content associated with said electronic information when said electronic information is accessed. 

1 19. The system according to claim 10, wherein said processor outputs selected audio 

2 content to be played through said output when a page of said electronic information is displayed. 

1 20. The system according to claim 19, wherein said processor automatically plays 

2 said selected audio content when said page is displayed. 

1 21. The system according to claim 1 0, further comprising: 

2 a communication link to transmit said audio content with its properties. 

1 22. The system according to claim 21, further comprising; 

2 a network connected to said communication link for receiving said audio content with 

3 properties, said network being accessible by other users. 

1 23 . The system according to claim 2 1 , further comprising: 

2 a receiving device of another user for receiving said audio content with properties, said 

3 receiving device receiving said audio content through one of a wired or wireless interface. 

1 24. The system according to claim 22, wherein said network further processes said 

2 audio content. 

1 25. The system according to claim 22, wherein said network includes a database for 

2 storing said audio content. 
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1 26. The system according to claim 22, wherein said network receives audio content 

2 without receiving said electronic information associated with said audio content. 

1 27. A user interface for displaying electronic information to a user comprising: 

2 a first display portion for displaying a portion of a document; and 

3 a second display portion for displaying a graphical indication that said document includes 

4 an audio annotation associated with said displayed portion of said document. 

1 28. The user interface according to claim 27, further comprising: 

2 a third display portion for displaying a non-audio annotation. 

1 29. The user interface according to claim 27, further comprising: 

2 a third display portion for displaying an indication that said audio annotation is being 

3 recorded or played back. 

1 30. The user interface according to claim 27, further comprising: 

2 a third display portion for displaying one of a document tape or a master tape. 

1 31. The user interface according to claim 27, further comprising: 

2 a third display portion for receiving a user input of a property or properties of said audio 

3 annotation. 

1 32. The user interface according to claim 27, wherein said audio annotation is 

2 recordable by said user. 

1 33. A process for recording an audio annotation comprising the steps of: 

2 displaying electronic information; 

3 receiving a user input; 

4 recording an audio annotation in response to said user input; and 
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5 associating said audio annotation with properties including a displayed portion of said 

6 electronic information. 

1 34. The process according to claim 33, further comprising the step of: 

2 storing said audio annotation prior to the association of said audio annotation with said 

3 displayed portion. 

1 35. The process according to claim 33, further comprising the step of: 

2 storing said audio annotation after the association of said audio annotation with said 



3 displayed portion. 

1 36. The process according to claim 33, wherein said recording step records all 

AS a, 

^ 2 ambient sounds. 

J 1 37. The process according to claim 33, wherein said recording step records only 

m 2 sounds above a predetermined threshold. 

jy i 38, The process according to claim 37, wherein said recording step records only a 

O 2 specific user's voice. 



fU i 39. The process according to claim 33, further comprising the step of: 

H 2 associating additional properties with said audio annotation at the start of recording of 

3 said audio annotation. 

1 40. The process according to claim 33, wherein one of said properties is a file 

2 position or document position of an item on said displayed portion of said electronic information. 

1 41. The process according to claim 33, wherein one of said properties is a start 

2 identification of said displayed portion of said electronic information. 

1 42. The process according to claim 33, further comprising the steps of: 

2 storing said audio annotation; and, 
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3 searching audio annotations including said audio annotation for at least one property 

4 matching a query. 

1 43. A process for playing audio annotations comprising the steps of: 

2 displaying a portion of electronic information; 

3 receiving a user input; 

4 retrieving audio annotations; 

5 assembling said audio annotations into an audio stream; and 

6 playing said audio stream. 

1 44. The process according to claim 43, further comprising the step of: 

2 waiting for a second user input prior to playing said audio stream. 

1 45. The process according to claim 43, further comprising the step of: 

2 playing once said audio stream is assembled. 

1 46. The process according to claim 43, wherein said user input is a text query. 

1 47. The process according to claim 43, wherein said user input is a voice query. 

1 48. The process according to claim 43, further comprising the steps of: 

2 altering the display of said portion to match a currently playing annotation in said audio 

3 stream. 

1 49. The process according to claim 48, wherein said altering step includes the steps 

2 of: 

3 comparing the length of said currently playing annotation with the starting identifications 

4 of displayable portions of said electronic information; and 

5 displaying the portion of said electronic information supporting the greater length of said 

6 currently playing annotation. 
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1 50. A process for playing audio annotations comprising the steps of: 

2 navigating to a page; 

3 retrieving at least one audio annotation associated with a page or associated with an item 

4 on a page; and 

5 playing said at least one audio annotation. 

1 51. The process according to claim 50, further comprising the step of: 

2 waiting for a user input prior to playing said audio annotation. 

1 52. The process according to claim 50, wherein said item on said page includes at 

2 least one of embedded notes, inked notes, highlights and underlining. 

1 53. The process according to claim 50, wherein said at least one audio annotation was 

2 previously retrieved and said retrieving step includes indexing said previously retrieved at least 

3 one audio annotation. 

1 54. The process according to claim 50, wherein said at least one audio annotation is 

2 the result of a newly executed query. 

1 55. A computer readable medium having a data structure stored thereon, said data 

2 structure comprising: 

3 a document; 

4 a link object; and 

5 audio content with at least one property, 

6 wherein said link object references said document and references said audio content. 

1 56. The data structure according to claim 55, wherein said property relates to the time 

2 said audio content started recording. 
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1 57. The data structure according to claim 55, wherein said property relates to the time 

2 said audio content stopped recording. 

1 58. The data structure according to claim 55, wherein said property relates to the 

2 length of recording of said audio content. 

1 59. The data structure according to claim 55, wherein said property relates to the 

2 author of the recording. 

1 60. The data structure according to claim 55, wherein said property relates to the start 

2 ID. 

1 61. The data structure according to claim 55, wherein said property relates to the stop 

02 ID. 

yi 62. The data structure according to claim 55, wherein said audio content is comprised 

!;j::2 of a plurality of audio clips. 

r\ 1 63. The data structure according to claim 62, wherein said audio clips are stored in a 

r%2 database. 

flj 1 64. The data structure according to claim 55, wherein said property is one of plurality 

C32 of properties and said properties are in a marked up language form. 
1 65. The data structure according to claim 64, wherein said properties are in XML. 

1 66. The data structure according to claim 55, wherein said audio content is stored 

2 within a document. 

1 67. The data structure according to claim 55, wherein said audio content is stored 

2 apart from a document. 
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1 68. The data structure according to claim 67, wherein said audio content is stored in a 

2 database with at least one property designating the position of viewed document relating to said 

3 audio content. 

1 69. The data structure according to claim 67, wherein said audio content is stored in a 

2 database and linked to a separate annotation document that stores the position of a viewed 

3 document relating to said audio content. 

1 70. A process for recording audio content comprising the steps of: 

2 navigating to a page of a document; 

3 recording said audio content; and 

O4 associating properties with said audio content such that retrieval of said audio content 

ijS positions said audio content after previously recorded audio content. 

jjfl 71. The process according to claim 70, wherein said audio content comprises audio 

[j2 clips and wherein said associating step includes a time property. 

p 1 72. The process according to claim 71, wherein said audio content and said 

fU2 previously recorded audio content is ordered at least by said time property. 

P 1 73. A process of searching audio clips comprising the steps of: 

2 inputting search terms or properties; 

3 searching said audio clips for said search terms or properties; and 

4 ordering audio clips detected by said searching step for output. 

1 74. The process according to claim 73, wherein said inputting step further comprises 

2 the steps of: 

3 receiving verbally delimited keywords; and 

4 converting said verbally delimited keywords into search terms or properties. 
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1 75 . A process for recording audio information comprising the steps of: 

2 recording audio signals as a first file; 

3 processing said file to extract audio clips; and 

4 storing said audio clips, 

5 wherein said processing separates the content of said first file into audio clips based on 

6 events. 

1 76. The process for recording according to claim 75, 

2 wherein said audio signals include speech, and 

3 wherein said events comprise at least one of short pauses in said speech, a pause of a 
C?4 predetermined length, and a user navigating away from a displayed page. 

^1 77. A process for associating audio notes and handwritten notes comprising the steps 

S 2 ° f; 

y 3 creating a handwritten note; 

Q4 associating a time at which said handwritten note was created with said handwritten note; 

fW 5 creating an audio note; and 

p 6 associating a time at which said audio note was created with said audio note, 

7 wherein, upon selection of said handwritten note, audio notes recorded at or near the time 

8 at which said handwritten note was created are located. 

1 78. The process according to claim 77, wherein locating said audio notes includes the 

2 step of querying a database for audio clips. 

1 79. The process according to claim 77, wherein locating said audio notes includes the 

2 step of searching a table. 
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1 80. The process according to claim 77, wherein locating said audio notes includes the 

2 step of searching a linked list. 

1 81. The process of claim 77, wherein said audio notes are comprised of audio clips in 

2 which each audio clip has a time of creation associated with each audio clip. 

1 82. The process according to claim 77, further comprising the step of: 

2 playing said audio notes. 

1 83. A process for playing audio notes comprising the steps of: 

2 displaying a first page of electronic information; 

3 playing audio notes associated with said first page; 

4 displaying a second page of electronic information; and, 

5 playing audio notes associated with said second page. 

1 84. The process according to claim 83, further comprising the step of receiving user 

2 input, 

3 wherein, in response to said user input, said second page is displayed. 

1 85. A process of recording audio notes comprising the steps of: 

2 displaying a first page of electronic information; 

3 recording a first set of audio notes; 

4 associating said first set of audio notes with said first page; 

5 displaying a second page of electronic information; 

6 recording a second set of audio notes; and 

7 associating said second set of audio notes with said second page. 

1 86. The process according to claim 85, further comprising the step of receiving user 

2 input, 
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3 wherein, in response to said user input, said second page is displayed. 

1 87. A process for editing audio notes comprising the steps of: 

2 querying a database for audio information; 

3 ordering said audio information into audio notes; and 

4 performing editing features on said audio notes. 

1 88. The process for editing audio notes according to claim 87, wherein said editing 

2 comprises at least one of the steps of: 

3 adding audio information; 

4 deleting audio information; and 

5 overwriting existing audio information. 
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